EDA - Pandas Profiling - Default of Credit Card Clients

Dataset statistics

Number of variables	25
Number of observations	20000
Missing cells	0
Missing cells (%)	0.0%
Duplicate rows	0
Duplicate rows (%)	0.0%
Total size in memory	3.8 MiB
Average record size in memory	200.0 B

Variable types

NUM	22
CAT	2
BOOL	1

Reproduction

Analysis started	2020-11-05 01:15:16.013152
Analysis finished	2020-11-05 01:18:10.262330
Duration	2 minutes and 54.25 seconds
Version	pandas-profiling v2.8.0
Command line	`pandas_profiling --config_file config.yaml [YOUR_FILE.csv]`
Download configuration	config.yaml

Warnings

`BILL_AMT2` is highly correlated with `BILL_AMT1` and 1 other fields	High correlation
`BILL_AMT1` is highly correlated with `BILL_AMT2`	High correlation
`BILL_AMT3` is highly correlated with `BILL_AMT2` and 1 other fields	High correlation
`BILL_AMT4` is highly correlated with `BILL_AMT3` and 2 other fields	High correlation
`BILL_AMT5` is highly correlated with `BILL_AMT4` and 1 other fields	High correlation
`BILL_AMT6` is highly correlated with `BILL_AMT4` and 1 other fields	High correlation
`PAY_AMT2` is highly skewed (γ1 = 30.58709063)	Skewed
`ID` has unique values	Unique
`PAY_0` has 9765 (48.8%) zeros	Zeros
`PAY_2` has 10448 (52.2%) zeros	Zeros
`PAY_3` has 10489 (52.4%) zeros	Zeros
`PAY_4` has 11148 (55.7%) zeros	Zeros
`PAY_5` has 11272 (56.4%) zeros	Zeros
`PAY_6` has 10646 (53.2%) zeros	Zeros
`BILL_AMT1` has 1330 (6.7%) zeros	Zeros
`BILL_AMT2` has 1697 (8.5%) zeros	Zeros
`BILL_AMT3` has 1957 (9.8%) zeros	Zeros
`BILL_AMT4` has 2122 (10.6%) zeros	Zeros
`BILL_AMT5` has 2373 (11.9%) zeros	Zeros
`BILL_AMT6` has 2749 (13.7%) zeros	Zeros
`PAY_AMT1` has 3597 (18.0%) zeros	Zeros
`PAY_AMT2` has 3724 (18.6%) zeros	Zeros
`PAY_AMT3` has 4129 (20.6%) zeros	Zeros
`PAY_AMT4` has 4407 (22.0%) zeros	Zeros
`PAY_AMT5` has 4544 (22.7%) zeros	Zeros
`PAY_AMT6` has 4940 (24.7%) zeros	Zeros

ID
Real number (ℝ_≥0)

UNIQUE

Distinct count	20000
Unique (%)	100.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	10000.5
Minimum	1
Maximum	20000
Zeros	0
Zeros (%)	0.0%
Memory size	156.2 KiB

Quantile statistics

Minimum	1
5-th percentile	1000.95
Q1	5000.75
median	10000.5
Q3	15000.25
95-th percentile	19000.05
Maximum	20000
Range	19999
Interquartile range (IQR)	9999.5

Descriptive statistics

Standard deviation	5773.647028
Coefficient of variation (CV)	0.577335836
Kurtosis	-1.2
Mean	10000.5
Median Absolute Deviation (MAD)	5000
Skewness	0
Sum	200010000
Variance	33335000

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
2047	1	< 0.1%
10912	1	< 0.1%
12947	1	< 0.1%
2708	1	< 0.1%
661	1	< 0.1%
6806	1	< 0.1%
4759	1	< 0.1%
19100	1	< 0.1%
17053	1	< 0.1%
8865	1	< 0.1%
Other values (19990)	19990	> 99.9%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
1	1	< 0.1%
2	1	< 0.1%
3	1	< 0.1%
4	1	< 0.1%
5	1	< 0.1%

Value	Count	Frequency (%)
20000	1	< 0.1%
19999	1	< 0.1%
19998	1	< 0.1%
19997	1	< 0.1%
19996	1	< 0.1%

LIMIT_BAL
Real number (ℝ_≥0)

Distinct count	76
Unique (%)	0.4%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	163301.184
Minimum	10000.0
Maximum	1000000.0
Zeros	0
Zeros (%)	0.0%
Memory size	156.2 KiB

Quantile statistics

Minimum	10000
5-th percentile	20000
Q1	50000
median	130000
Q3	230000
95-th percentile	420000
Maximum	1000000
Range	990000
Interquartile range (IQR)	180000

Descriptive statistics

Standard deviation	128746.7033
Coefficient of variation (CV)	0.7884003049
Kurtosis	0.6077365416
Mean	163301.184
Median Absolute Deviation (MAD)	80000
Skewness	1.029815015
Sum	3266023680
Variance	1.65757136e+10

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
50000	2354	11.8%
20000	1355	6.8%
30000	1175	5.9%
80000	1063	5.3%
200000	990	5.0%
150000	731	3.7%
100000	699	3.5%
180000	653	3.3%
360000	574	2.9%
60000	567	2.8%
Other values (66)	9839	49.2%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
10000	339	1.7%
16000	1	< 0.1%
20000	1355	6.8%
30000	1175	5.9%
40000	154	0.8%

Value	Count	Frequency (%)
1000000	1	< 0.1%
800000	2	< 0.1%
750000	4	< 0.1%
740000	1	< 0.1%
720000	1	< 0.1%

SEX
Categorical

Distinct count	2
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	156.2 KiB

2	12281
1	7719

Common Values
Length

Value	Count	Frequency (%)
2	12281	61.4%
1	7719	38.6%

Length

Max length	1
Median length	1
Mean length	1
Min length	1

EDUCATION
Real number (ℝ_≥0)

Distinct count	7
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	1.83695
Minimum	0
Maximum	6
Zeros	9
Zeros (%)	< 0.1%
Memory size	156.2 KiB

Quantile statistics

Minimum	0
5-th percentile	1
Q1	1
median	2
Q3	2
95-th percentile	3
Maximum	6
Range	6
Interquartile range (IQR)	1

Descriptive statistics

Standard deviation	0.7695416215
Coefficient of variation (CV)	0.4189235534
Kurtosis	1.878845998
Mean	1.83695
Median Absolute Deviation (MAD)	1
Skewness	0.9010035439
Sum	36739
Variance	0.5921943072

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
2	9451	47.3%
1	7113	35.6%
3	3191	16.0%
5	151	0.8%
4	57	0.3%
6	28	0.1%
0	9	< 0.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	9	< 0.1%
1	7113	35.6%
2	9451	47.3%
3	3191	16.0%
4	57	0.3%

Value	Count	Frequency (%)
6	28	0.1%
5	151	0.8%
4	57	0.3%
3	3191	16.0%
2	9451	47.3%

MARRIAGE
Categorical

Distinct count	4
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	156.2 KiB

2	10702
1	9033
3	232
0	33

Common Values
Length

Value	Count	Frequency (%)
2	10702	53.5%
1	9033	45.2%
3	232	1.2%
0	33	0.2%

Length

Max length	1
Median length	1
Mean length	1
Min length	1

AGE
Real number (ℝ_≥0)

Distinct count	55
Unique (%)	0.3%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	35.33325
Minimum	21
Maximum	79
Zeros	0
Zeros (%)	0.0%
Memory size	156.2 KiB

Quantile statistics

Minimum	21
5-th percentile	23
Q1	28
median	34
Q3	41
95-th percentile	53
Maximum	79
Range	58
Interquartile range (IQR)	13

Descriptive statistics

Standard deviation	9.210658839
Coefficient of variation (CV)	0.2606796386
Kurtosis	0.06718029386
Mean	35.33325
Median Absolute Deviation (MAD)	6
Skewness	0.7401311192
Sum	706665
Variance	84.83623625

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
29	1068	5.3%
27	974	4.9%
28	943	4.7%
30	910	4.5%
26	871	4.4%
25	814	4.1%
24	786	3.9%
33	776	3.9%
34	767	3.8%
31	766	3.8%
Other values (45)	11325	56.6%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
21	46	0.2%
22	410	2.1%
23	661	3.3%
24	786	3.9%
25	814	4.1%

Value	Count	Frequency (%)
79	1	< 0.1%
75	1	< 0.1%
73	2	< 0.1%
72	1	< 0.1%
71	2	< 0.1%

PAY_0
Real number (ℝ)

ZEROS

Distinct count	11
Unique (%)	0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	0.02145
Minimum	-2
Maximum	8
Zeros	9765
Zeros (%)	48.8%
Memory size	156.2 KiB

Quantile statistics

Minimum	-2
5-th percentile	-2
Q1	-1
median	0
Q3	0
95-th percentile	2
Maximum	8
Range	10
Interquartile range (IQR)	1

Descriptive statistics

Standard deviation	1.121094439
Coefficient of variation (CV)	52.26547499
Kurtosis	3.21004583
Mean	0.02145
Median Absolute Deviation (MAD)	1
Skewness	0.8208641883
Sum	429
Variance	1.25685274

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	9765	48.8%
-1	3886	19.4%
1	2569	12.8%
2	1890	9.4%
-2	1585	7.9%
3	202	1.0%
4	58	0.3%
8	17	0.1%
5	13	0.1%
6	8	< 0.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-2	1585	7.9%
-1	3886	19.4%
0	9765	48.8%
1	2569	12.8%
2	1890	9.4%

Value	Count	Frequency (%)
8	17	0.1%
7	7	< 0.1%
6	8	< 0.1%
5	13	0.1%
4	58	0.3%

PAY_2
Real number (ℝ)

ZEROS

Distinct count	11
Unique (%)	0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	-0.1042
Minimum	-2
Maximum	8
Zeros	10448
Zeros (%)	52.2%
Memory size	156.2 KiB

Quantile statistics

Minimum	-2
5-th percentile	-2
Q1	-1
median	0
Q3	0
95-th percentile	2
Maximum	8
Range	10
Interquartile range (IQR)	1

Descriptive statistics

Standard deviation	1.204124103
Coefficient of variation (CV)	-11.5558935
Kurtosis	1.730356002
Mean	-0.1042
Median Absolute Deviation (MAD)	0
Skewness	0.8319999703
Sum	-2084
Variance	1.449914856

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	10448	52.2%
-1	4114	20.6%
2	2742	13.7%
-2	2341	11.7%
3	228	1.1%
4	61	0.3%
5	20	0.1%
1	19	0.1%
7	17	0.1%
6	9	< 0.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-2	2341	11.7%
-1	4114	20.6%
0	10448	52.2%
1	19	0.1%
2	2742	13.7%

Value	Count	Frequency (%)
8	1	< 0.1%
7	17	0.1%
6	9	< 0.1%
5	20	0.1%
4	61	0.3%

PAY_3
Real number (ℝ)

ZEROS

Distinct count	11
Unique (%)	0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	-0.1363
Minimum	-2
Maximum	8
Zeros	10489
Zeros (%)	52.4%
Memory size	156.2 KiB

Quantile statistics

Minimum	-2
5-th percentile	-2
Q1	-1
median	0
Q3	0
95-th percentile	2
Maximum	8
Range	10
Interquartile range (IQR)	1

Descriptive statistics

Standard deviation	1.210659157
Coefficient of variation (CV)	-8.882312231
Kurtosis	2.532713318
Mean	-0.1363
Median Absolute Deviation (MAD)	0
Skewness	0.935031521
Sum	-2726
Variance	1.465695595

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	10489	52.4%
-1	4032	20.2%
2	2655	13.3%
-2	2547	12.7%
3	150	0.8%
4	59	0.3%
7	27	0.1%
6	20	0.1%
5	15	0.1%
1	4	< 0.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-2	2547	12.7%
-1	4032	20.2%
0	10489	52.4%
1	4	< 0.1%
2	2655	13.3%

Value	Count	Frequency (%)
8	2	< 0.1%
7	27	0.1%
6	20	0.1%
5	15	0.1%
4	59	0.3%

PAY_4
Real number (ℝ)

ZEROS

Distinct count	11
Unique (%)	0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	-0.19735
Minimum	-2
Maximum	8
Zeros	11148
Zeros (%)	55.7%
Memory size	156.2 KiB

Quantile statistics

Minimum	-2
5-th percentile	-2
Q1	-1
median	0
Q3	0
95-th percentile	2
Maximum	8
Range	10
Interquartile range (IQR)	1

Descriptive statistics

Standard deviation	1.16806301
Coefficient of variation (CV)	-5.918738334
Kurtosis	3.857599321
Mean	-0.19735
Median Absolute Deviation (MAD)	0
Skewness	1.066045653
Sum	-3947
Variance	1.364371196

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	11148	55.7%
-1	3772	18.9%
-2	2722	13.6%
2	2095	10.5%
3	135	0.7%
4	51	0.3%
7	41	0.2%
5	27	0.1%
6	5	< 0.1%
8	2	< 0.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-2	2722	13.6%
-1	3772	18.9%
0	11148	55.7%
1	2	< 0.1%
2	2095	10.5%

Value	Count	Frequency (%)
8	2	< 0.1%
7	41	0.2%
6	5	< 0.1%
5	27	0.1%
4	51	0.3%

PAY_5
Real number (ℝ)

ZEROS

Distinct count	10
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	-0.2339
Minimum	-2
Maximum	8
Zeros	11272
Zeros (%)	56.4%
Memory size	156.2 KiB

Quantile statistics

Minimum	-2
5-th percentile	-2
Q1	-1
median	0
Q3	0
95-th percentile	2
Maximum	8
Range	10
Interquartile range (IQR)	1

Descriptive statistics

Standard deviation	1.142478032
Coefficient of variation (CV)	-4.884472132
Kurtosis	4.002151329
Mean	-0.2339
Median Absolute Deviation (MAD)	0
Skewness	1.043625791
Sum	-4678
Variance	1.305256053

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	11272	56.4%
-1	3774	18.9%
-2	2830	14.1%
2	1876	9.4%
3	130	0.7%
4	64	0.3%
7	41	0.2%
5	9	< 0.1%
6	3	< 0.1%
8	1	< 0.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-2	2830	14.1%
-1	3774	18.9%
0	11272	56.4%
2	1876	9.4%
3	130	0.7%

Value	Count	Frequency (%)
8	1	< 0.1%
7	41	0.2%
6	3	< 0.1%
5	9	< 0.1%
4	64	0.3%

PAY_6
Real number (ℝ)

ZEROS

Distinct count	10
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	-0.2614
Minimum	-2
Maximum	8
Zeros	10646
Zeros (%)	53.2%
Memory size	156.2 KiB

Quantile statistics

Minimum	-2
5-th percentile	-2
Q1	-1
median	0
Q3	0
95-th percentile	2
Maximum	8
Range	10
Interquartile range (IQR)	1

Descriptive statistics

Standard deviation	1.167063898
Coefficient of variation (CV)	-4.464666786
Kurtosis	3.378433112
Mean	-0.2614
Median Absolute Deviation (MAD)	0
Skewness	0.9887823395
Sum	-5228
Variance	1.362038142

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	10646	53.2%
-1	4041	20.2%
-2	3069	15.3%
2	2014	10.1%
3	140	0.7%
7	32	0.2%
4	32	0.2%
6	15	0.1%
5	9	< 0.1%
8	2	< 0.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-2	3069	15.3%
-1	4041	20.2%
0	10646	53.2%
2	2014	10.1%
3	140	0.7%

Value	Count	Frequency (%)
8	2	< 0.1%
7	32	0.2%
6	15	0.1%
5	9	< 0.1%
4	32	0.2%

BILL_AMT1
Real number (ℝ)

HIGH CORRELATION
ZEROS

Distinct count	15918
Unique (%)	79.6%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	50022.29625
Minimum	-165580.0
Maximum	964511.0
Zeros	1330
Zeros (%)	6.7%
Memory size	156.2 KiB

Quantile statistics

Minimum	-165580
5-th percentile	0
Q1	3688.25
median	22541
Q3	65061.75
95-th percentile	194282
Maximum	964511
Range	1130091
Interquartile range (IQR)	61373.5

Descriptive statistics

Standard deviation	71498.06441
Coefficient of variation (CV)	1.429323917
Kurtosis	10.32326362
Mean	50022.29625
Median Absolute Deviation (MAD)	21845
Skewness	2.704989067
Sum	1000445925
Variance	5111973214

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	1330	6.7%
390	152	0.8%
780	58	0.3%
316	51	0.3%
326	50	0.2%
2500	37	0.2%
396	28	0.1%
2400	27	0.1%
1050	19	0.1%
-200	19	0.1%
Other values (15908)	18229	91.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-165580	1	< 0.1%
-15308	1	< 0.1%
-14386	1	< 0.1%
-9802	1	< 0.1%
-9095	1	< 0.1%

Value	Count	Frequency (%)
964511	1	< 0.1%
630458	1	< 0.1%
621749	1	< 0.1%
610723	1	< 0.1%
604019	1	< 0.1%

BILL_AMT2
Real number (ℝ)

HIGH CORRELATION
ZEROS

Distinct count	15638
Unique (%)	78.2%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	48149.92915
Minimum	-33350.0
Maximum	983931.0
Zeros	1697
Zeros (%)	8.5%
Memory size	156.2 KiB

Quantile statistics

Minimum	-33350
5-th percentile	0
Q1	3147
median	21532
Q3	62435.75
95-th percentile	187924.35
Maximum	983931
Range	1017281
Interquartile range (IQR)	59288.75

Descriptive statistics

Standard deviation	69443.17584
Coefficient of variation (CV)	1.442227996
Kurtosis	11.32318576
Mean	48149.92915
Median Absolute Deviation (MAD)	21136
Skewness	2.785568925
Sum	962998583
Variance	4822354671

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	1697	8.5%
390	144	0.7%
316	58	0.3%
326	52	0.3%
780	46	0.2%
2500	34	0.2%
2400	29	0.1%
396	28	0.1%
-200	23	0.1%
1050	20	0.1%
Other values (15628)	17869	89.3%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-33350	1	< 0.1%
-30000	1	< 0.1%
-26214	1	< 0.1%
-24704	1	< 0.1%
-24702	1	< 0.1%

Value	Count	Frequency (%)
983931	1	< 0.1%
743970	1	< 0.1%
646770	1	< 0.1%
605943	1	< 0.1%
597793	1	< 0.1%

BILL_AMT3
Real number (ℝ)

HIGH CORRELATION
ZEROS

Distinct count	15384
Unique (%)	76.9%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	45728.7874
Minimum	-157264.0
Maximum	1664089.0
Zeros	1957
Zeros (%)	9.8%
Memory size	156.2 KiB

Quantile statistics

Minimum	-157264
5-th percentile	0
Q1	2720
median	20160
Q3	58668.5
95-th percentile	179964.75
Maximum	1664089
Range	1821353
Interquartile range (IQR)	55948.5

Descriptive statistics

Standard deviation	67151.05479
Coefficient of variation (CV)	1.468463491
Kurtosis	25.96669101
Mean	45728.7874
Median Absolute Deviation (MAD)	19770
Skewness	3.285701752
Sum	914575748
Variance	4509264160

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	1957	9.8%
390	176	0.9%
780	51	0.3%
316	51	0.3%
326	44	0.2%
396	28	0.1%
2500	26	0.1%
2400	26	0.1%
200	22	0.1%
416	18	0.1%
Other values (15374)	17601	88.0%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-157264	1	< 0.1%
-61506	1	< 0.1%
-34041	1	< 0.1%
-20320	1	< 0.1%
-15910	1	< 0.1%

Value	Count	Frequency (%)
1664089	1	< 0.1%
693131	1	< 0.1%
597415	1	< 0.1%
578971	1	< 0.1%
548020	1	< 0.1%

BILL_AMT4
Real number (ℝ)

HIGH CORRELATION
ZEROS

Distinct count	15059
Unique (%)	75.3%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	41465.52835
Minimum	-170000.0
Maximum	891586.0
Zeros	2122
Zeros (%)	10.6%
Memory size	156.2 KiB

Quantile statistics

Minimum	-170000
5-th percentile	0
Q1	2309.5
median	18889.5
Q3	51123.5
95-th percentile	166661.95
Maximum	891586
Range	1061586
Interquartile range (IQR)	48814

Descriptive statistics

Standard deviation	61660.90664
Coefficient of variation (CV)	1.487040177
Kurtosis	11.98792725
Mean	41465.52835
Median Absolute Deviation (MAD)	18326.5
Skewness	2.872809794
Sum	829310567
Variance	3802067407

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	2122	10.6%
390	162	0.8%
780	68	0.3%
316	52	0.3%
326	44	0.2%
150	32	0.2%
396	27	0.1%
2400	27	0.1%
2500	25	0.1%
416	22	0.1%
Other values (15049)	17419	87.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-170000	1	< 0.1%
-81334	1	< 0.1%
-34503	1	< 0.1%
-24303	1	< 0.1%
-20320	1	< 0.1%

Value	Count	Frequency (%)
891586	1	< 0.1%
628699	1	< 0.1%
569034	1	< 0.1%
542653	1	< 0.1%
530672	1	< 0.1%

BILL_AMT5
Real number (ℝ)

HIGH CORRELATION
ZEROS

Distinct count	14727
Unique (%)	73.6%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	39526.67165
Minimum	-37594.0
Maximum	927171.0
Zeros	2373
Zeros (%)	11.9%
Memory size	156.2 KiB

Quantile statistics

Minimum	-37594
5-th percentile	0
Q1	1718.75
median	18132
Q3	49529.25
95-th percentile	160874
Maximum	927171
Range	964765
Interquartile range (IQR)	47810.5

Descriptive statistics

Standard deviation	59309.32739
Coefficient of variation (CV)	1.500488782
Kurtosis	12.42749486
Mean	39526.67165
Median Absolute Deviation (MAD)	17713.5
Skewness	2.8824658
Sum	790533433
Variance	3517596315

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	2373	11.9%
390	165	0.8%
316	62	0.3%
780	57	0.3%
326	44	0.2%
150	39	0.2%
396	28	0.1%
2400	27	0.1%
416	24	0.1%
2500	23	0.1%
Other values (14717)	17158	85.8%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-37594	1	< 0.1%
-36156	1	< 0.1%
-28335	1	< 0.1%
-23003	1	< 0.1%
-20753	1	< 0.1%

Value	Count	Frequency (%)
927171	1	< 0.1%
551702	1	< 0.1%
547880	1	< 0.1%
505473	1	< 0.1%
503914	1	< 0.1%

BILL_AMT6
Real number (ℝ)

HIGH CORRELATION
ZEROS

Distinct count	14339
Unique (%)	71.7%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	38175.69155
Minimum	-339603.0
Maximum	961664.0
Zeros	2749
Zeros (%)	13.7%
Memory size	156.2 KiB

Quantile statistics

Minimum	-339603
5-th percentile	0
Q1	1193.75
median	16995.5
Q3	48672.25
95-th percentile	157758.05
Maximum	961664
Range	1301267
Interquartile range (IQR)	47478.5

Descriptive statistics

Standard deviation	58707.21876
Coefficient of variation (CV)	1.537816772
Kurtosis	13.82804934
Mean	38175.69155
Median Absolute Deviation (MAD)	16679.5
Skewness	2.945456668
Sum	763513831
Variance	3446537534

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	2749	13.7%
390	133	0.7%
316	65	0.3%
150	59	0.3%
780	56	0.3%
326	40	0.2%
396	28	0.1%
-18	22	0.1%
2400	22	0.1%
416	22	0.1%
Other values (14329)	16804	84.0%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-339603	1	< 0.1%
-150953	1	< 0.1%
-51443	1	< 0.1%
-51183	1	< 0.1%
-45734	1	< 0.1%

Value	Count	Frequency (%)
961664	1	< 0.1%
699944	1	< 0.1%
568638	1	< 0.1%
527711	1	< 0.1%
527566	1	< 0.1%

PAY_AMT1
Real number (ℝ_≥0)

ZEROS

Distinct count	6067
Unique (%)	30.3%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	5521.0682
Minimum	0.0
Maximum	505000.0
Zeros	3597
Zeros (%)	18.0%
Memory size	156.2 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	967.75
median	2084
Q3	5000
95-th percentile	18163.55
Maximum	505000
Range	505000
Interquartile range (IQR)	4032.25

Descriptive statistics

Standard deviation	15250.37482
Coefficient of variation (CV)	2.762214534
Kurtosis	192.6437216
Mean	5521.0682
Median Absolute Deviation (MAD)	1916.5
Skewness	11.10137038
Sum	110421364
Variance	232573932.2

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	3597	18.0%
2000	909	4.5%
3000	573	2.9%
5000	465	2.3%
1500	353	1.8%
4000	300	1.5%
1000	265	1.3%
10000	258	1.3%
2500	199	1.0%
6000	184	0.9%
Other values (6057)	12897	64.5%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	3597	18.0%
1	6	< 0.1%
2	12	0.1%
3	8	< 0.1%
4	10	0.1%

Value	Count	Frequency (%)
505000	1	< 0.1%
405016	1	< 0.1%
368199	1	< 0.1%
302000	1	< 0.1%
300000	1	< 0.1%

PAY_AMT2
Real number (ℝ_≥0)

SKEWED
ZEROS

Distinct count	5922
Unique (%)	29.6%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	5746.19355
Minimum	0.0
Maximum	1684259.0
Zeros	3724
Zeros (%)	18.6%
Memory size	156.2 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	740.75
median	2000
Q3	5000
95-th percentile	18667.25
Maximum	1684259
Range	1684259
Interquartile range (IQR)	4259.25

Descriptive statistics

Standard deviation	21518.62324
Coefficient of variation (CV)	3.744848315
Kurtosis	1948.43083
Mean	5746.19355
Median Absolute Deviation (MAD)	1941
Skewness	30.58709063
Sum	114923871
Variance	463051145.9

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	3724	18.6%
2000	892	4.5%
3000	571	2.9%
5000	484	2.4%
1000	453	2.3%
1500	382	1.9%
4000	263	1.3%
10000	202	1.0%
6000	187	0.9%
1200	173	0.9%
Other values (5912)	12669	63.3%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	3724	18.6%
1	9	< 0.1%
2	15	0.1%
3	15	0.1%
4	7	< 0.1%

Value	Count	Frequency (%)
1684259	1	< 0.1%
580464	1	< 0.1%
415552	1	< 0.1%
401003	1	< 0.1%
385228	1	< 0.1%

PAY_AMT3
Real number (ℝ_≥0)

ZEROS

Distinct count	5500
Unique (%)	27.5%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	4734.48815
Minimum	0.0
Maximum	896040.0
Zeros	4129
Zeros (%)	20.6%
Memory size	156.2 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	322
median	1593
Q3	4054.5
95-th percentile	15304.8
Maximum	896040
Range	896040
Interquartile range (IQR)	3732.5

Descriptive statistics

Standard deviation	15823.31417
Coefficient of variation (CV)	3.342138299
Kurtosis	639.0253309
Mean	4734.48815
Median Absolute Deviation (MAD)	1593
Skewness	17.74716555
Sum	94689763
Variance	250377271.3

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	4129	20.6%
1000	837	4.2%
2000	826	4.1%
3000	559	2.8%
5000	511	2.6%
1500	304	1.5%
4000	249	1.2%
10000	214	1.1%
2500	160	0.8%
6000	158	0.8%
Other values (5490)	12053	60.3%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	4129	20.6%
1	9	< 0.1%
2	17	0.1%
3	10	0.1%
4	12	0.1%

Value	Count	Frequency (%)
896040	1	< 0.1%
417588	1	< 0.1%
371718	1	< 0.1%
338394	1	< 0.1%
332809	1	< 0.1%

PAY_AMT4
Real number (ℝ_≥0)

ZEROS

Distinct count	5293
Unique (%)	26.5%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	4725.79775
Minimum	0.0
Maximum	497000.0
Zeros	4407
Zeros (%)	22.0%
Memory size	156.2 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	237.75
median	1496.5
Q3	4000
95-th percentile	15847.15
Maximum	497000
Range	497000
Interquartile range (IQR)	3762.25

Descriptive statistics

Standard deviation	15180.46154
Coefficient of variation (CV)	3.212253749
Kurtosis	213.8612252
Mean	4725.79775
Median Absolute Deviation (MAD)	1496.5
Skewness	11.72346383
Sum	94515955
Variance	230446412.6

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	4407	22.0%
1000	929	4.6%
2000	800	4.0%
3000	594	3.0%
5000	545	2.7%
1500	301	1.5%
4000	274	1.4%
10000	214	1.1%
500	183	0.9%
6000	176	0.9%
Other values (5283)	11577	57.9%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	4407	22.0%
1	16	0.1%
2	13	0.1%
3	11	0.1%
4	13	0.1%

Value	Count	Frequency (%)
497000	1	< 0.1%
432130	1	< 0.1%
400046	1	< 0.1%
331788	1	< 0.1%
330982	1	< 0.1%

PAY_AMT5
Real number (ℝ_≥0)

ZEROS

Distinct count	5248
Unique (%)	26.2%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	4758.7926
Minimum	0.0
Maximum	417990.0
Zeros	4544
Zeros (%)	22.7%
Memory size	156.2 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	216
median	1500
Q3	4000
95-th percentile	15794.75
Maximum	417990
Range	417990
Interquartile range (IQR)	3784

Descriptive statistics

Standard deviation	15447.36965
Coefficient of variation (CV)	3.246069108
Kurtosis	177.201995
Mean	4758.7926
Median Absolute Deviation (MAD)	1500
Skewness	11.1072561
Sum	95175852
Variance	238621229.1

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	4544	22.7%
1000	889	4.4%
2000	861	4.3%
3000	623	3.1%
5000	551	2.8%
1500	290	1.5%
4000	259	1.3%
10000	208	1.0%
500	167	0.8%
2500	153	0.8%
Other values (5238)	11455	57.3%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	4544	22.7%
1	15	0.1%
2	8	< 0.1%
3	7	< 0.1%
4	8	< 0.1%

Value	Count	Frequency (%)
417990	1	< 0.1%
388071	1	< 0.1%
379267	1	< 0.1%
332000	1	< 0.1%
330982	1	< 0.1%

PAY_AMT6
Real number (ℝ_≥0)

ZEROS

Distinct count	5227
Unique (%)	26.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	5080.15935
Minimum	0.0
Maximum	528666.0
Zeros	4940
Zeros (%)	24.7%
Memory size	156.2 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	10
median	1407
Q3	4000
95-th percentile	17398.1
Maximum	528666
Range	528666
Interquartile range (IQR)	3990

Descriptive statistics

Standard deviation	17306.82153
Coefficient of variation (CV)	3.406747769
Kurtosis	176.7194765
Mean	5080.15935
Median Absolute Deviation (MAD)	1407
Skewness	10.69589634
Sum	101603187
Variance	299526071.6

Histogram

Histogram with fixed size bins (bins=10)

Value	Count	Frequency (%)
0	4940	24.7%
1000	873	4.4%
2000	862	4.3%
3000	575	2.9%
5000	539	2.7%
1500	319	1.6%
4000	282	1.4%
10000	234	1.2%
500	174	0.9%
6000	153	0.8%
Other values (5217)	11049	55.2%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	4940	24.7%
1	15	0.1%
2	6	< 0.1%
3	10	0.1%
4	7	< 0.1%

Value	Count	Frequency (%)
528666	1	< 0.1%
527143	1	< 0.1%
403500	1	< 0.1%
372495	1	< 0.1%
345293	1	< 0.1%

default.payment.next.month
Boolean

Distinct count	2
Unique (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	156.2 KiB

0	15442
1	4558

Frequency Table

Value	Count	Frequency (%)
0	15442	77.2%
1	4558	22.8%

Pearson's r

The Pearson's correlation coefficient (r) is a measure of linear correlation between two variables. It's value lies between -1 and +1, -1 indicating total negative linear correlation, 0 indicating no linear correlation and 1 indicating total positive linear correlation. Furthermore, r is invariant under separate changes in location and scale of the two variables, implying that for a linear function the angle to the x-axis does not affect r.

To calculate r for two variables X and Y, one divides the covariance of X and Y by the product of their standard deviations.

Spearman's ρ

The Spearman's rank correlation coefficient (ρ) is a measure of monotonic correlation between two variables, and is therefore better in catching nonlinear monotonic correlations than Pearson's r. It's value lies between -1 and +1, -1 indicating total negative monotonic correlation, 0 indicating no monotonic correlation and 1 indicating total positive monotonic correlation.

To calculate ρ for two variables X and Y, one divides the covariance of the rank variables of X and Y by the product of their standard deviations.

Kendall's τ

Similarly to Spearman's rank correlation coefficient, the Kendall rank correlation coefficient (τ) measures ordinal association between two variables. It's value lies between -1 and +1, -1 indicating total negative correlation, 0 indicating no correlation and 1 indicating total positive correlation.

To calculate τ for two variables X and Y, one determines the number of concordant and discordant pairs of observations. τ is given by the number of concordant pairs minus the discordant pairs divided by the total number of pairs.

Phik (φk)

Phik (φk) is a new and practical correlation coefficient that works consistently between categorical, ordinal and interval variables, captures non-linear dependency and reverts to the Pearson correlation coefficient in case of a bivariate normal input distribution. There is extensive documentation available here.

Cramér's V (φc)

Cramér's V is an association measure for nominal random variables. The coefficient ranges from 0 to 1, with 0 indicating independence and 1 indicating perfect association. The empirical estimators used for Cramér's V have been proved to be biased, even for large samples. We use a bias-corrected measure that has been proposed by Bergsma in 2013 that can be found here.

Count
Matrix

First rows

	ID	LIMIT_BAL	SEX	EDUCATION	MARRIAGE	AGE	PAY_0	PAY_2	PAY_3	PAY_4	PAY_5	PAY_6	BILL_AMT1	BILL_AMT2	BILL_AMT3	BILL_AMT4	BILL_AMT5	BILL_AMT6	PAY_AMT1	PAY_AMT2	PAY_AMT3	PAY_AMT4	PAY_AMT5	PAY_AMT6	default.payment.next.month
0	1	20000.0	2	2	1	24	2	2	-1	-1	-2	-2	3913.0	3102.0	689.0	0.0	0.0	0.0	0.0	689.0	0.0	0.0	0.0	0.0	1
1	2	120000.0	2	2	2	26	-1	2	0	0	0	2	2682.0	1725.0	2682.0	3272.0	3455.0	3261.0	0.0	1000.0	1000.0	1000.0	0.0	2000.0	1
2	3	90000.0	2	2	2	34	0	0	0	0	0	0	29239.0	14027.0	13559.0	14331.0	14948.0	15549.0	1518.0	1500.0	1000.0	1000.0	1000.0	5000.0	0
3	4	50000.0	2	2	1	37	0	0	0	0	0	0	46990.0	48233.0	49291.0	28314.0	28959.0	29547.0	2000.0	2019.0	1200.0	1100.0	1069.0	1000.0	0
4	5	50000.0	1	2	1	57	-1	0	-1	0	0	0	8617.0	5670.0	35835.0	20940.0	19146.0	19131.0	2000.0	36681.0	10000.0	9000.0	689.0	679.0	0
5	6	50000.0	1	1	2	37	0	0	0	0	0	0	64400.0	57069.0	57608.0	19394.0	19619.0	20024.0	2500.0	1815.0	657.0	1000.0	1000.0	800.0	0
6	7	500000.0	1	1	2	29	0	0	0	0	0	0	367965.0	412023.0	445007.0	542653.0	483003.0	473944.0	55000.0	40000.0	38000.0	20239.0	13750.0	13770.0	0
7	8	100000.0	2	2	2	23	0	-1	-1	0	0	-1	11876.0	380.0	601.0	221.0	-159.0	567.0	380.0	601.0	0.0	581.0	1687.0	1542.0	0
8	9	140000.0	2	3	1	28	0	0	2	0	0	0	11285.0	14096.0	12108.0	12211.0	11793.0	3719.0	3329.0	0.0	432.0	1000.0	1000.0	1000.0	0
9	10	20000.0	1	3	2	35	-2	-2	-2	-2	-1	-1	0.0	0.0	0.0	0.0	13007.0	13912.0	0.0	0.0	0.0	13007.0	1122.0	0.0	0

Last rows

	ID	LIMIT_BAL	SEX	EDUCATION	MARRIAGE	AGE	PAY_0	PAY_2	PAY_3	PAY_4	PAY_5	PAY_6	BILL_AMT1	BILL_AMT2	BILL_AMT3	BILL_AMT4	BILL_AMT5	BILL_AMT6	PAY_AMT1	PAY_AMT2	PAY_AMT3	PAY_AMT4	PAY_AMT5	PAY_AMT6	default.payment.next.month
19990	19991	150000.0	2	2	1	32	-1	2	-1	0	0	0	3204.0	430.0	29425.0	30437.0	31123.0	28997.0	0.0	29425.0	2000.0	1528.0	1518.0	2000.0	0
19991	19992	70000.0	2	2	1	33	1	2	0	0	0	0	51900.0	49307.0	48214.0	27572.0	26810.0	20541.0	0.0	1500.0	1300.0	1505.0	1000.0	1000.0	0
19992	19993	240000.0	2	2	1	34	-1	-1	0	0	-1	-1	626.0	1921.0	20740.0	21274.0	888.0	360.0	1921.0	19000.0	2624.0	888.0	360.0	360.0	0
19993	19994	250000.0	2	1	1	49	-1	-1	-2	-2	-1	0	1104.0	0.0	0.0	0.0	3000.0	1500.0	0.0	0.0	0.0	3000.0	0.0	3212.0	0
19994	19995	440000.0	2	2	1	41	0	0	0	0	0	0	348397.0	356586.0	366049.0	262697.0	267922.0	274502.0	14006.0	19077.0	9518.0	9576.0	11083.0	12010.0	0
19995	19996	130000.0	2	2	1	40	0	0	0	0	0	0	133559.0	129869.0	118032.0	95953.0	73970.0	107785.0	5400.0	6950.0	4600.0	4000.0	2000.0	2300.0	0
19996	19997	60000.0	2	2	1	37	0	0	0	0	0	0	59462.0	60866.0	54007.0	52089.0	29397.0	29110.0	3000.0	2570.0	2202.0	1200.0	1100.0	1100.0	1
19997	19998	290000.0	2	2	1	41	-1	-1	-2	-1	0	-1	2025.0	0.0	0.0	9194.0	9194.0	399.0	0.0	0.0	9194.0	0.0	399.0	9290.0	0
19998	19999	150000.0	2	2	1	41	0	0	-1	-1	-1	-1	4474.0	3881.0	1207.0	1617.0	0.0	620.0	3000.0	2306.0	2610.0	0.0	620.0	0.0	0
19999	20000	240000.0	2	2	1	37	-1	2	-1	-1	-1	0	1769.0	842.0	14015.0	0.0	1317.0	566.0	0.0	14015.0	0.0	1317.0	0.0	0.0	0

Overview

Variables

Interactions

Correlations

Pearson's r

Spearman's ρ

Kendall's τ

Phik (φk)

Cramér's V (φc)

Missing values

Sample

First rows

Last rows